{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Structured Representations\n",
    "\n",
    "This model shows a method for constructing structured representations using semantic pointers (high dimensional vector representations). It uses a convolution network to bind two Semantic Pointers and a Sum network to cojoin two semantic pointers. \n",
    "\n",
    "**Note: This model can be simplified if built using the spa (semantic pointer architecture) package in Nengo 2.0. This method is shown in the last few sections of this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Setup the environment\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "import nengo\n",
    "from nengo.spa import Vocabulary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the Model\n",
    "This model has parameters as described in the book, with the ensembles having 20 dimensions and 300 neurons each. You will use the '`nengo.networks.CircularConvolution`' class in Nengo 2.0 to compute the convolution (or binding) of two semantic pointers `A` and `B`. \n",
    "\n",
    "Since the collection of named vectors in a space forms a kind of \"vocabulary\" as described in the book, you will create a vocabulary to build structured representations out of it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dim = 20           # Number of dimensions \n",
    "num_neurons = 300  # Number of neurons in each ensemble\n",
    "\n",
    "# Creating a vocabulary\n",
    "rng = np.random.RandomState(0)\n",
    "vocab = Vocabulary(dimensions=dim, rng=rng)\n",
    "\n",
    "model = nengo.Network(label = 'Structured Representation')\n",
    "with model:\n",
    "    # Input - Get the raw vectors for the pointers using `vocab['A'].v`\n",
    "    inputA = nengo.Node(output=vocab['A'].v)\n",
    "    inputB = nengo.Node(output=vocab['B'].v)\n",
    "    \n",
    "    # Ensembles with 300 neurons and 20 dimensions\n",
    "    A = nengo.Ensemble(num_neurons, dimensions=dim)      # Represents inputA\n",
    "    B = nengo.Ensemble(num_neurons, dimensions=dim)      # Represents inputB\n",
    "    C = nengo.Ensemble(num_neurons, dimensions=dim)      # Represents the convolution of A and B \n",
    "    Sum =  nengo.Ensemble(num_neurons, dimensions=dim)   # Represents the sum of A and B \n",
    "    \n",
    "    # Creating the circular convolution network with 70 neurons per dimension\n",
    "    Bind = nengo.networks.CircularConvolution(70, dimensions=dim)\n",
    "    \n",
    "    # Connecting the input to ensembles A and B\n",
    "    nengo.Connection(inputA, A)\n",
    "    nengo.Connection(inputB, B)  \n",
    "    \n",
    "    # Projecting ensembles A and B to the Bind network\n",
    "    nengo.Connection(A, Bind.A)\n",
    "    nengo.Connection(B, Bind.B)\n",
    "    nengo.Connection(Bind.output, C)\n",
    "    \n",
    "    # Projecting ensembles A and B to the Sum ensemble\n",
    "    nengo.Connection(A, Sum)\n",
    "    nengo.Connection(B, Sum) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Add Probes to Collect Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Anything that is probed will collect the data it produces over time, allowing you to analyze and visualize it later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "with model:\n",
    "    A_probe = nengo.Probe(A, synapse=0.03)\n",
    "    B_probe = nengo.Probe(B, synapse=0.03)\n",
    "    C_probe = nengo.Probe(C, synapse=0.03)\n",
    "    Sum_probe = nengo.Probe(Sum, synapse=0.03)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Set `C` to equal the convolution of `A` with `B` in the vocabulary. \n",
    "# This will be your ground-truth to test the accuracy of the neural network.\n",
    "vocab.add('C', vocab.parse('A * B'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "In order to run the model without nengo_gui, you have to create a simulator. Then, you can run that simulator over and over again without affecting the original model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "with nengo.Simulator(model) as sim:  # Create the simulator\n",
    "    sim.run(1.0)                     # Run it for one second"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot the Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize=(14, 3))\n",
    "plt.subplot(1, 3, 1)\n",
    "plt.plot(sim.trange(), sim.data[A_probe])\n",
    "plt.title(\"Decoded Ensemble A\")\n",
    "\n",
    "plt.subplot(1, 3, 2)\n",
    "plt.plot(sim.trange(), sim.data[B_probe])\n",
    "plt.title(\"Decoded Ensemble B\")\n",
    "\n",
    "plt.figure(figsize=(14, 3))\n",
    "plt.subplot(1, 3, 1)\n",
    "plt.plot(sim.trange(), sim.data[Sum_probe])\n",
    "plt.title(\"Sum (cojoin)\")\n",
    "\n",
    "plt.subplot(1, 3, 2)\n",
    "plt.plot(sim.trange(), sim.data[C_probe])\n",
    "plt.title(\"C (convolution)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The graphs above show the value of individual components of their respective ensembles. They show the same information as the \"value\" graphs in the Interactive Plots in Nengo 1.4 GUI as described in the book."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analyze the results\n",
    "\n",
    "The Interactive Plots in Nengo 1.4 GUI can also be used to see the \"semantic pointer\" graph of an ensemble as described in the book. You can create similarity graphs to get the same information by plotting the similarity between the semantic pointer represented by an ensemble and all the semantic pointers in the vocabulary. The dot product is a common measure of similarity between semantic pointers, since it approximates the cosine similarity when the semantic pointer lengths are close to one.\n",
    "\n",
    "For this model, you can plot the exact convolution of `A` and `B` (given by `vocab.parse('A * B')`), and the result of the neural convolution (given by `sim.data[C_probe]`).\n",
    "\n",
    "Both the dot product and the exact cosine similarity can be computed with '`nengo.spa.similarity`'. Normally, this function will compute the dot product, but setting '`normalize=True`' normalizes all vectors so that the exact cosine similarity is computed instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "plt.plot(sim.trange(), nengo.spa.similarity(sim.data[C_probe], vocab))\n",
    "plt.legend(vocab.keys, loc=4)\n",
    "plt.xlabel(\"t [s]\")\n",
    "plt.ylabel(\"dot product\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above plot shows that the neural output is much closer to `C = A * B` than to either `A` or `B`, suggesting that the network is correctly computing the convolution. The dot product between the neural output and `C` is not exactly one due in large part to the fact that the length of `C` is not exactly one. Using cosine similarity, this magnitude difference can be neglected. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "plt.plot(sim.trange(), nengo.spa.similarity(sim.data[C_probe], vocab, normalize=True))\n",
    "plt.legend(vocab.keys, loc=4)\n",
    "plt.xlabel(\"t [s]\")\n",
    "plt.ylabel(\"cosine similarity\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see that the cosine similarity between the neural output vectors and `C` is almost exactly one, demonstrating that the neural population is quite accurate in computing the convolution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the model using the `spa` package"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will build this model again, using the spa (semantic pointer architecture) package built into Nengo 2.0. You will see that using the spa package considerably simplifies model construction and visualization through nengo_gui."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import nengo\n",
    "import nengo.spa as spa\n",
    "from nengo.spa import Vocabulary\n",
    "import numpy as np\n",
    "\n",
    "D = 32  # the dimensionality of the vectors\n",
    "\n",
    "# Creating a vocabulary\n",
    "rng = np.random.RandomState(0)\n",
    "vocab = Vocabulary(dimensions=D, rng=rng)\n",
    "vocab.add('C', vocab.parse('A * B'))\n",
    "\n",
    "model = spa.SPA(label=\"structure\", vocabs=[vocab])\n",
    "with model:\n",
    "    model.A = spa.State(D)\n",
    "    model.B = spa.State(D)\n",
    "    model.C = spa.State(D, feedback=1)\n",
    "    model.Sum = spa.State(D)\n",
    "\n",
    "    actions = spa.Actions(\n",
    "        'C = A * B',\n",
    "        'Sum = A',\n",
    "        'Sum = B'\n",
    "    )\n",
    "\n",
    "    model.cortical = spa.Cortical(actions)\n",
    "    \n",
    "    # Input\n",
    "    model.input = spa.Input(A='A', B='B')    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the model in nengo_gui"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the nengo_gui visualizer to run and visualize the model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from nengo_gui.ipython import IPythonViz\n",
    "IPythonViz(model, \"ch4-structure.py.cfg\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Press the play button in the visualizer to run the simulation. You should see the \"semantic pointer cloud\" graphs as shown in the figure below.\n",
    "\n",
    "The graphs `A` and `B` show the semantic pointer representations in objects `A` and `B` respectively. Graphs labelled `C` show the result of the convolution operation (left - shows the semantic pointer representation in object `C`, right - shows the similarity with the vectors in the vocabulary). The graphs labelled `Sum` show the sum of A and B as represented by the object `Sum` (left - shows the semantic pointer representation in Sum, right - shows high similarity with vecotors A and B)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from IPython.display import Image\n",
    "Image(filename='ch4-structure.png')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
